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In October 2007, the Wiseonsin Poliey Researeh Institute released a research report on 
public school choice and parental involvement, Fixing the Milwaukee Public Schools: The Limits 
of Parent-Driven Reform} Beeause the report elaims that public school choice and parental 
involvement have little effeet on student performance in Milwaukee Public Schools, a surprising 
conclusion from a think tank that has traditionally supported public and private school choice, 
the report received signifieant news eoverage and editorial comment, partieularly from those 
opposed to school choice programs that include private sehool options, sueh as the Milwaukee 
Parental Choice Program. Headlines ineluded; 

“The failure of school choice,” Capital Times (Madison, Wis.) 

“Choice may not improve sehools, study says,” Milwaukee Journal Sentinel 

“School choice doesn’t necessarily add up,” The Kansas City Sta/ 

“Whoops, we goofed: School choice doesn’t work like its supporters promised. Gulp. 

Now what?” Milwaukee Magazine^ 

The Utah Demoeratie Party also ineluded the study’s results in an October press release 
just before Utah citizens went to the polls to vote on a statewide private school choice plan.^ The 
release described it as a “study released Tuesday in Milwaukee that suggests school choice isn’t 
a powerful tool for driving edueational improvement.” 

Unfortunately, most of this coverage and eommentary uneritically aecepted the report’s 
conelusions about publie school choice — and, even worse, applied them more broadly to private 
school choice programs. But, as this analysis finds, the WPRI report is fatally flawed, 
undermining both its claims about public school choice and any implications for private school 
choice in Milwaukee or elsewhere. Among the flaws: 

1. The WPRI report studies only public school choice. The Milwaukee Parental Choiee 

Program is not ineluded. Beeause public school choice plans differ fundamentally from 
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choice plans that include private sehools, the results from the WPRI report eannot be 
generalized, even though its authors have suggested otherwise. 

2. There is strong empirieal evidence (from far more methodologically rigorous research) 
that competition in Milwaukee and elsewhere that includes private school choice spurs 
improvement in publie sehools and leads to greater student performanee and more 
satisfied parents among families who choose their ehildren’s schools. 

3. The WPRI report does not actually study Milwaukee families. It uses data collected by 
the U.S. Department of Edueation for a nationwide study and attempts to extrapolate 
those results to Milwaukee, a proeess the Department of Education advises against — with 
good reason. 

4. At no time does the WPRI report formally test the relationship or impact of public school 
choice or parental involvement on student performance, or any other outeome measure. 

It merely surmises how mueh parental involvement there might be in Milwaukee and 
how parents might make deeisions about their ehildren’s sehooling and then notes the 
poor performanee of student performance in Milwaukee Public Schools, assuming but 
never proving a causal link between them. 

5. The report’s theoretical understanding and definitions of parental involvement and how 
parents deeide on their children’s sehooling appears critically out of step with prior 
research findings. Moreover, the creation of the study’s variables appears empirically 
unjustified. 

Taken together, these limitations in WPRI’s study make its eonclusions utterly meaningless. 

The WPRI Report 

WPRl’s report attempts to determine the effect of public school choice and parental 
involvement on Milwaukee Publie Schools. The theory is that public school choice will create 
competition for students, improving school effectiveness, efficieney, and ultimately student 
outeomes. Similarly, parental involvement engages parents as partners in the educational 
process, thereby harnessing their eonsiderable influence over their children’s time, attitudes, 
effort, and engagement, which should also contribute to improved student outcomes. 

WPRI finds only 10 pereent of Milwaukee parents aet as what the author defines as 
“ideal consumers” when deeiding about their ehildren’s schooling. And the author claims that 
parents do not appear suffieiently involved in their ehildren’s edueation. Therefore, WPRI 
asserts, Milwaukee Public Schools are likely not sensing any pressure of an educational 
marketplace or enjoying enough parental engagement to affeet significant change. 
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What the WPRI Report Does Not Say About Private School Choice 

Key Finding: The WPRI report examines only public school choice. Because choice plans that 
include private options differ substantively from public school choice plans, the WPRI report 
results cannot be generalized to school choice in its broader definition. 

Media coverage about the WPRI report and use of it by special interest groups has 
conveyed a message that the WPRI report’s findings apply to all forms of school choice, 
including private school choice/ That is false. The WPRI report only considered public school 
choice, which includes open enrollment plans within and between districts, magnet schools, and 
charter schools. The Milwaukee Parental Choice Program, which includes private school 
options, was not included. For that reason alone, the WPRI report findings cannot be generalized 
to school choice more broadly. 

In addition, the difference between school choice plans that allow private options and 
those limited only to public options is so large as to make generalizations from one to the other 
spurious. The theory of educational marketplaces that include private options holds, in part, that 
competition created as a result of students leaving the public system will spur public schools to 
improve and to innovate as they work harder to retain and attract students (and therefore dollars). 

When choice is limited only to public options, public schools do not “feel the heat” of a 

o 

genuine marketplace so as to spur change, despite the WPRI report’s inferences to the contrary. 
This is so for at least four reasons: 

1 . In intra-district choice the student (and the money) never leaves the district. 

2. In inter-district choice, equilibrium often is created over time in that districts initially lose 
students but will gain others.^ 

3. As discussed in greater detail below, open enrollment (i.e., intra- and inter-district choice) 
fails to provide parents with attractive or viable options, severely reducing the incentive 
to take advantage of choice. Consider Minnesota’s statewide open enrollment plan, in 
which students, as of 1987, can attend any public school of their choosing. Statewide, 
fewer than 2% of students exercise the right to attend a school outside their home 
district. 

4. In most localities, the capacity of charter schools and magnet schools remains too limited 
to accommodate a critical mass of students to leave their assigned schools. For example, 
a little more than 10% of Milwaukee students attend charter schools,^' even though the 
charter law has been in effect since 1993. 

In contrast to the WPRI report, rigorous research on choice plans that do include private 
options show public schools improve, students perform better in choice schools, and parents are 
more satisfied. An example of such a work specifically on Milwaukee schools comes in the 

12 

form of a paper written by New York Federal Reserve Bank economist, Rajashri Chakrabarti. 
Using data from 1987 to 2002, and a robust, empirical research design, Chakrabarti showed how 
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private school competition led to an improvement in public schools in Milwaukee. It is yet 
another paper that follows on the heels of rigorous, empirical studies by a multitude of authors 
demonstrating that authentic school choice — plans that include private providers — leads to 
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positive outcomes both individually and collectively. 

Not only does the WPRI report have no implications for private school choice, its 
conclusions about public school choice are undermined by its poor methodology. 

Data and Methodology 

Key Finding: The most critical flaws with the WPRI report lie in its methods, which fatally 
undermine its findings. These flaws range from technical issues to critical data use and 
inference issues. It is no exaggeration to say that the report ’s methodological shortcomings 
make its findings utterly meaningless. 

The data in the WPRI report came from the 2003 Parent and Family Involvement in 
Education survey, which is part of the National Household Education Surveys Program (NHES) 
funded by the U.S. Department of Education. The NHES provides descriptive data on the 
educational activities of the U.S. population. It covers learning at all ages through multiple 
surveys, one of which measures parental involvement.'"^ 

Erom the NHES database, WPRI pulled several variables representing its definition of an 
“ideal consumer” and each parent’s level of involvement, combined them mathematically, and 
inferred those results from a national sample to Milwaukee using a weighting system based on 
four demographic variables in the Census Bureau’s 2005 American Community Survey (ACS). 
The ACS is an annual survey designed to identify changes in an area’s population and give an 
up-to-date statistical picture every year, not just once in ten years with the decennial census.'^ 

More specifically, the ACS was used to determine weights by examining the educational 
attainment of parents, race and ethnicity of students, household composition (single-parent vs. 
two-parent), and mother’s employment status for Milwaukee residents. Those weights were then 
applied to the data in the NHES database when calculating percentages for each of the 
aforementioned choice and involvement variables. The theory behind such weighting is stated 
on page 13 of the WPRI report: “[PJrevious research has shown a variety of demographic 
variables to be correlated with important aspects of both parental choice and parent involvement 
in education.” 

In other words, the theory is that people act differently based on the selected 
demographic characteristics; therefore, WPRI adjusted the NHES data using weights to reflect 
the demographic composition of Milwaukee. However, the NHES data were not from 
Milwaukee specifically. Rather, the NHES data included respondents from all over the country, 
some of whom were likely from Milwaukee but most of whom were not. 

One of the implications of such a procedure is that in a nationwide sample like this the 
participants likely lack any direct experience with various choice options. Open enrollment (i.e., 
inter and intra district choice) is not available in all states, and for many with such policies. 
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district participation is voluntary/^ Likewise, charter sehools are not available in 10 states, and 
even in those states with charter laws, enabling legislation is not universally strong,'^ whieh 
limits the number of schools that can or do form. Finally, the presenee of magnet sehools is very 
uneven aeross the United States. A little more than half of the states even have magnet sehools, 
and in those that do the numbers range widely from one or two in the entire state to more than 
400 in a few others. Therefore, the NHES data used in this study likely ineludes large numbers 
of people for whom public school choice is limited or entirely irrelevant. 

Methodological Flaw: Sampling Error 

Key Finding: In all empirical research sampling is one of the most important elements of 
ensuring valid and reliable results. The WPRI report ’s sampling procedure, using a national 
data set rather than data directly from Milwaukee residents, makes its findings meaningless. 

Unlike numerous studies of ehoice and parental involvement that gather data directly 
from the population of interest,^^ the WPRI report does not gather data directly from Milwaukee 
residents. Instead, it infers from a national sample in the NHES to Milwaukee, a totally 
inappropriate procedure. In fact, an NHES website with information for researehers written in 
EAQ style states: 

* Can I use NHES data for state- or eounty-level analysis? 

The NHES data are intended to generate national and regional (e.g. North, East, South, 
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Midwest, West) estimates. 

The NHES technical manuals likewise state that the NHES, including the Parent and 

Eamily Involvement sub-survey, is designed to make inferenees up to the population not down to 

2 1 

city and county levels. 

Although it would be an ill-advised procedure, it would have been possible for the WPRI 
report to access data from Milwaukee residents in the NHES using data listed as “restricted.” 

Eike most if not all of the large federal data sets, the NHES is available to researchers in a public 
use form, which removes means of potentially identifying survey respondents. However, 
researehers may apply for aceess to the restricted use data, which would have enabled the WPRI 
to sort the NHES data by zip code and look only at data from Milwaukee respondents. 

Still, this alternative would not solve other problems with WPRI’s method. And it would 
come with its own set of significant limitations, specifically related to issues of sample weights 
that arise with complex sampling proeedures present in surveys like the NHES.^^ Thus, the 
value in raising this issue is not necessarily as a eourse of reeommended aetion but as a further 
example of the WPRI report’s inadequate and uninformed researeh. 

Acknowledging that collecting original data is difficult, time eonsuming, and often 
prohibitively expensive, a more advisable course of action could have been to use data drawn 
specifically from Milwaukee by other researchers. For example, John Witte and Christopher 
Thom collected data from Milwaukee specifically on public school choice and parental 
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involvement. Although the WPRI report makes no mention of attempting to aecess these data, 
it would be surprising to learn that Witte and Thom refused aceess given the research custom of 
making data available to others. 

Methodological Flaw: No Cause-and-Effect Measured 

Key Finding: WPRI fails to formally test the effect of public school choice and parental 
involvement on MPS performance, making its cause-and-ejfect conclusions unfounded. 

From the WPRI report, the media coverage surrounding it, and the use of the report by 
special interest groups, one is left with the impression that the report examines the relationships 
between public school choice, parental involvement, and some outcome, or dependent measure 
related to performance in the Milwaukee Public Schools. In other words, the report gives the 
impression that its methods test for a causal link between public school choice and parental 
involvement on one side of the equation and the performance of MPS on the other. This is false. 

In fact, in this study there is no dependent variable, or effect. Only possible causes are 
measured, with no attempt to see if those presumed causes actually produce a change in the 
outcome variable, MPS performance. At no time does the report ever formally test the 
relationship between choice and involvement on any outcome, despite cause-and-effect 
conclusions like, “Taken as a whole, these numbers indicate significant limits on the capacity of 
public school choice and parental involvement to improve school quality and student 
performance within MPS.”^"^ 

Moreover, the exact research question driving the study is not entirely clear. Much of the 
study seems to focus on the relationships between public school choice, parental involvement, 
and student performance, but late in the report this head-scratcher appears: 

As noted in the text, previous research has shown a variety of demographic variables to 
be correlated with important aspects of both parental choice and parent involvement in 
education. The intent of this study was to explore the impact of those variables on choice 

25 

and involvement in a specific education context — the MPS system. 

If this is so, then that makes public school choice and parental involvement outcome, or 
dependent variables, unless, of course, the intent is to examine them as mediating variables 
between the demographics and student performance. If so, the study’s design is even more 
inadequate. Unfortunately, the study’s research question is so muddled one is left only with 
guesses. 

Methodological Flaw: Improper Creation of Variables 

Key Finding: The variables used to measure public school choice and parental involvement lack 
any empirical footing. There is little reason to trust the validity ofWPRFs measures. 

In the WPRI report, both public school choice and parental involvement were constructed 
variables using multiple indicators from the NHES data. This is a common and accepted 




Fatally Flawed 
Institute for Justice 
Page 7 

procedure, sinee phenomena like choice and involvement are multidimensional and should not 
be represented by a single indieator. However, the standard procedure in ereating variables from 

multiple indicators using large datasets like NHES is to utilize a proeess called factor analysis — 

26 

something the WPRI report apparently did not use. 

Faetor analysis takes a large number of indieators or variables and statistically determines 
the interrelations among them. This provides a researcher with an idea of which indicators “hang 
together” or share some eommonality so as to combine them into one variable. Indieators that do 
not share the required amount of eommonality are not eombined. 

Consider the parental involvement variables used in this study for students aged 10 to 14: 

1 . estabhshed rules regarding television viewing and completion of homework; 

2. discussed with the child his or her experiences at school “often” (as opposed to 
“sometimes” or “never”); 

3. participated in at least one of a series of activities with the child in the preceding month; 

4. regularly checked to see if the student had completed his or her homework; and 

5. assisted with homework at least one or two days per week. 

A factor analysis might show that indicators four and live share mueh in eommon with 
eaeh other but not with any of the other variables. Moreover, none of the other three may show 
any eommonality. As a result, it would be inappropriate to eombine all live indieators into one 
variable. This is hypothetieal, of eourse, since we have no idea how the indicators relate to one 
another; the WPRI report provides no record of measuring that. 

This is a partieularly signifieant shorteoming given ineonsistent findings in prior 
research, discussed below. Yet, even if there were some a priori reason to believe eertain 
indieators should be combined, it is common to test that relationship in the data. This is done 
through eonfirmatory factor analysis in structural equation modeling or at the very least by using 
Cronbach’s Alpha to test how strongly indieators correlate. But, again, the WPRI report appears 
to have done none of these. 

Theory and Prior Research 

Drawing valid and reliable research conclusions depends on a sound theoretieal 
foundation and a valid representation of those theories in the variables used in the study. 
Unfortunately, the WPRI report falls short. 

Wrongly Defining the “Right” Way to Choose 

Key Finding: The WPRI report purports to define the “right” way parents should choose a 
school. But there is no one “correct ” or “rational ” way to choose a school any more than there 
is one “right” type of child. 

Like WPRI, prior researehers have examined how and why parents ehoose a sehool. The 
how refers to the decision-making process parents use in choosing their ehild’s school, 
particularly information gathering processes and types of information sourees. The why refers to 
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the features of a potential school that compel parents to choose it, such as academic record, 
school safety, and location. 

In the WPRI report, the how and why are represented by two variables: whether parents 
consider at least two schools in the choice process and whether they utilize performance- 
based/academic criteria. In so doing, the report appears to define what constitutes the “righf ’ and 
“wrong” way parents should choose their child’s school. 

This sounds like something right out of rational choice theory, where a rational actor is 
one who, when faced with a decision, engages in a search for the best information before 
deciding.^^ Yet, the context of parental decision-making (the how) is far more complex than the 
rational calculations represented by the report’s measure. According to prior research findings, 
“To make decisions regarding their children’s education, parents will rely on their personal 
values and subjective desired goals of education, as well as others within their social and 
professional networks to collect information.”^^ The latter source, social networks (i.e., word of 
mouth), is a particularly common source of information.^'^ 

Indeed, 79% percent of public school parents report relying on their social networks of 
friends, neighbors, and other parents to inform their decision, followed by talks with teachers 
(59%) and school visits (43%). More “rational” sources of information, including published 
school achievement scores (9%), the school newsletter (3%), or media reports (3%), are used by 
only a small percentage of parents. Moreover, nearly half of public school parents indicate they 
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sent their child to their assigned school without first seeking information. 

Other research indicates the why is likewise not as “rational” as the WPRI report 
theorizes, in that parental choices may not necessarily focus on academic outcomes. Rather, 
parents may select schools out of safety concerns or based on student demographics. Indeed, 
numerous studies indicate parents often choose schools for reasons other than academic 
outcomes.^^ 

For example, 50% of public school parents indicate Proximity to Their Home as the most 
important factor in choosing a school, followed by Academic Reputation (28%), The Teachers 
(24%), The Principal (22%), and Teaching Style (21%).^"' In Milwaukee specifically, 
educational quality, discipline, the general atmosphere, and location of the schools are among the 

35 

more important factors for parents. 

Are such decisions “irrational?” Do parents use the “wrong” reasons for choosing 
schools? According to the WPRI report, the apparent answer is “yes.” Yet, given the 
circumstances public school parents face in Milwaukee, choosing to remain in their 
neighborhood public schools or using non-academic sources and reasons for their choice could 
be entirely rational, given the academic quality in the MPS, the “sameness” of public schools, 
and the probability of gaining entry into a truly different public school. 

Little Real Choice Among Public Schools 

Key Finding: WPRI wrongly assumes that parents can freely choose among public schools and 
that there are meaningful choices to make. But in public school choice, staying in the assigned 
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school without looking at other options or not considering academic information when deciding 
makes when the only choice is more of the same and when institutional barriers make it unlikely 
that choices will be honored. 

As the WPRI report indicates, MPS, “like many of its big-city counterparts in other 
states, continues to suffer from poor student performance.”^^ Using reading, math, and science 
state assessment scores, the report demonstrates that MPS students lag far behind the rest of the 
state, and that gap only grows as students proceed through school. Assuming, of course, students 
proceed through school. As WPRI points out, “Only 68 percent of MPS high school students 
avoided dropping out and successfully earned their diploma. The comparable figure for the rest 
of the state was 91 percent.” 
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Moreover, as several authors argue, differences between public schools are scant. 

Indeed, in public school choice specifically, there are few differences between choice schools 
and comparison schools based on curricular content, teacher variables, procedures, or 
distinctiveness. Thus, compared to choice programs that include private schools options 
(which the WPRI study does not include), public school choice offers parents little in the way of 
authentic choices between truly distinctive curricular or programmatic offerings or academic 
outputs. 

Finally, Milwaukee parents may wish to enroll in charter, magnet, or other public schools 
in and outside of the district, but doing so is not as easy the WPRI report seems to imply. 

Charter schools enroll students based on a combination of space available (which is typically 
quite limited), district preference, and lottery systems, and magnet schools may select students 
based on criteria such as test scores, teacher recommendations, auditions, interviews, and 
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grades. 



Open enrollment, too, comes with limitations. Although all Wisconsin students may 
choose to attend a school outside of their home district, under Wisconsin Statute 1 18.51(5) 
school districts can reject enrollment applications for a host of reasons, ranging from space 
limitations to a child’s disability. Districts also have the discretion to cap the number of students 
they will allow to transfer into their schools, and no transportation is provided to transferring 
students. 

In the face of few authentic choices among consistently poor performing schools and the 
limitations inherent in enrollment procedures, it would be entirely rational for many Milwaukee 
parents not to even try to enroll their child in a different public school. This, then, could create 
the appearance of a low use of public school choice as described in the WPRI report. Moreover, 
defining the “right” choice as one in which only academic factors (itself a vague and limiting 
construct) are considered fails to take into consideration the complexity involved in such a 
decision. Given the uniqueness inherent within each child and the diverse familial circumstances 
represented in a large city like Milwaukee, academic factors will not be the only consideration by 
parents and rationally may not even be the primary concern."^' Taken together, these theoretical 
shortcomings undermine the report’s ability to draw the strong conclusions it does about public 
school choice. 
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Unfounded Assumptions About Parental Involvement 

Key Finding: The WPRI report ’s measures of parental involvement lack the necessary research- 
based consensus to make the sweeping conclusions it does. 

WPRI’s theoretical assumptions and shortcomings related to parental involvement do not 
contribute to an accurate picture of the degree of involvement, which, in turn, weakens the 
report’s conclusions. 

The theoretical limitations begin with the very definition of parental involvement. 

Despite the vast literature on this topic, few agree on what constitutes effective or meaningful 
involvement."^^ Because of the lack of scientific rigor in the research related to this issue less is 
known about parental involvement than commonly assumed, which could, and likely does, lead 
to unrealistic expectations on the part of researchers and policymakers about the potential for 
school improvement as a result of involvement and even how to define it. 

The WPRI report defines parental involvement in two ways — home-based and school- 
based. Each is constructed from multiple variables from the NHES database, but no research is 
cited that would indicate these variables are influential in improving academic performance of 
students or schools. Although some of these variables have been used in other studies,"^^ their 
effects have proven rather inconsistent."^"^ Therefore, drawing conclusions about their possible 
effect as an agent of change remains speculation. 

Indeed, the report acknowledges but only cursorily so just how uneven the research is on 
parental involvement."^^ Certainly some empirical studies have shown evidence of a positive 
effect from parental involvement,"^^ but others have found little, if any, such measurable effect."^^ 
It is not even clear if involvement is significant across all grade levels."^^ 

Some authors even report a negative relationship between parental involvement and 
academic achievement."^^ In fact, some of the variables used in the WPRI report have been 
shown to negatively influence student performance.^'^ Unfortunately, the variable that has 
consistently shown to have a positive effect, parental expectations,^' was not included in the 
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WPRI report, even though a measure of this variable was available in the database. 

Other Causes of Low Parental Involvement 

Key Finding: The WPRI report never bothers to ask why parental involvement is allegedly so 
low. The problem often begins with the school. 

The WPRI report’s conclusions are further undermined by the rather poor performance of 
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public schools in facilitating parental involvement. Schools have not engaged fully in 
involving parents meaningfully or granted parents access to opportunities for involvement in 
school affairs.^"' Parent-school partnerships are commonly established on the professionals’ 
terms, thereby creating barriers for parents. This happens because school personnel typically 
conceptualize, plan, and implement parental involvement programs without considering parents’ 
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interests. As a result, knowledge of parental needs and wishes are presumed, and parental 
eomplianee is taken for granted. 

Not surprisingly, parental involvement initiatives beeome meaningless to parents. This 
meaninglessness is further highlighted as parents eome to diseover they are shut out from 
deeision-making in the school. One of the most engaging types of parental involvement is 
including parents in school decisions, hut “This typically has not been a role played by most 
teachers. Instead, school personnel feel threatened by such parental involvement and resist 
including parents in decision-making.^^ 

Unfortunately, the report’s comparably extended discussion of parental involvement 
requirements in the No Child Left Behind Act of 2001 (NCLB) do nothing to ameliorate such 
findings. Although NCLB requires schools to institute programs designed to facilitate greater 
parental involvement, research on formal parental involvement programs has not been 
encouraging.^^ Moreover, recent research specifically on NCLB and parental involvement casts 
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serious doubt on the idea of substantive change as a result of NCLB. 

Therefore, the WPRI report’s inferences concerning parental involvement are founded 
upon uneven if not empirically unsubstantiated assumptions. Moreover, it never bothers to 
consider why involvement levels are so low, one answer of which includes the posture taken by 
schools toward involvement. In the face of such weaknesses, drawing firm conclusions about the 
effect of parental involvement as an intervention on MPS is reckless. 

Conclusion 

As evidenced by the scores of other studies on public school choice and parental 
involvement, the WPRI report’s weakness is not in asking unimportant questions but in drawing 
such sweeping (and ultimately meaningless) conclusions based on insufficient and erroneous 
theoretical assumptions and methods. To conclude as boldly as the report does that “these 
numbers indicate significant limits on the capacity of public school choice and parental 
involvement to improve school quality and student performance within MPS”^^ requires a well- 
grounded research study of complex design and analysis far beyond the rather simplistic 
approach taken by the WPRI report. 

The fatal flaws in the WPRI report not only undermine its conclusions about public 
school choice, they also make it impossible to draw any broader conclusions about school choice 
programs involving private schools, such as the Milwaukee Parental Choice Program. The poor 
quality of the WPRI report, and the unfortunate way it has been used in the media and by 
opponents, is a case study in the need for sound research to inform public policy debates. 
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